In the recent years, the computer image understanding has wide and profound applications in intelligence traffic, satellite remote sensing, machine vision, image analysis of medical treatment, Internet image search and etc. As its extension, the image holistic scene understanding is more complex and integrated than basic image scene understanding task. In this paper, the basic framework for image understanding, the researching implication and value, typical models for image holistic scene understanding were summarized. The four typical holistic scene understanding models were introduced, and the model frameworks were thoroughly compared. At last, some research insufficiency and future direction in image holistic scene understanding were presented, which pointed out some new insights for the further research in this area.